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Abstract 

How effective are interdomain routing protocols, such as the Border Gateway Protocol, at 
routing packets? Theoretical analyses have attempted to answer this question by ignoring the 
packets and instead focusing upon protocol stability. To study stability, it suffices to model only 
the control plane (which determines the routing graph) - an approach taken in the Stable Paths 
Problem. To analyse packet routing, it requires modelling the interactions between the control 
plane and the forwarding plane (which determines when packets are forwarded), and our first 
contribution is to introduce such a model. We then examine the effectiveness of packet routing 
in this model for the broad class next-hop preferences with filtering. Here each node v has a 
filtering list T>(v) consisting of nodes it does not want its packets to route through. Acceptable 
paths (those that avoid nodes in the filtering list) are ranked according to the next-hop, that 
is, the neighbour of v that the path begins with. On the negative side, we present a strong 
inapproximability result. For filtering lists of cardinality at most one, given a network in which 
an equilibrium is guaranteed to exist, it is NP-hard to approximate the maximum number of 
packets that can be routed to within a factor of 0(n 1 ~ e ), for any constant e > 0. On the positive 
side, we give algorithms to show that in two fundamental cases every packet will eventually route 
with probability one. The first case is when each node's filtering list contains only itself, that is, 
T>{y) = {v}. Moreover, with positive probability every packet will be routed before the control 
plane reaches an equilibrium. The second case is when all the filtering lists are empty, that is, 
T>(v) = 0. Thus, with probability one packets will route even when the nodes don't care if their 
packets cycle! Furthermore, with probability one every packet will route even when the control 
plane has no equilibrium at all. To our knowledge, these are the first results to guarantee the 
possibility that all packets get routed without stability. These positive results are tight - for 
the general case of filtering lists of cardinality one, it is not possible to ensure that every packet 
will eventually route. 



1 Introduction 

The Internet is a union of subnetworks called domains or Autonomous Systems (ASes). The inter- 
domain routing protocol used in the Internet today is called the Border Gateway Protocol (BGP), 
and it works as follows [TTj. For destination r and router v , each neighbouring router of v announces 
to v the route to r that it has chosen and from amongst these announced routes, v chooses the route 
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V{v) that it ranks highest. The router v then in turn announces to its neighbouring routers its 
routing path 'P(v). This process continues until an equilibrium is reached in which each router has 
chosen a route and for each router v, no neighbour of v announces a route that v would rank higher 
than its currently routing path. The ranking of routes at a router depends on a number of route 
attributes such as which neighbour announced the route, how long the route is, and which domains 
the route traverses. In fact, the ranking of routes at v is a function of u's traffic engineering goals 
as well as the Service Level Agreements (SLAs), that is, the economic contracts v has made with 
its neighbours. 

It is well known that BGP can be thought of as a game [1] and that BGP as a game may 
have no Nash equilibrium |18| [5]. There is now a vast literature studying the conditions under 
which BGP will or will not have an equilibrium (for example [HE]). It has been shown that in a 
BGP instance, the absence of a structure known as a dispute wheel implies that the BGP instance 
will have a unique equilibrium There have been a number of papers analysing the worst-case 
convergence time of BGP instances having no dispute wheel [SI [21 [16]. There have also been many 
experimental papers measuring BGP convergence times |12} [6] and papers offering modifications 
to BGP with the goal of speeding up convergence [151 ITT] - 

However, BGP convergence is only a step towards the ultimate goal of successfully delivering 
packets to the destination. In fact, routers perform operations simultaneously on two basic levels: 
(1) on the control plane (i.e., where BGP exchanges routing information with other routers as 
described above) and (2) on the forwarding plane where routers use the routing information from 
BGP to forward packets to neighbouring routers towards the packets' ultimate destinations. That 
is, packets are being forwarded during the time that the control plane is attempting to settle on an 
equilibrium. As mentioned, there has been a large theoretical study on the possibility or speed of 
convergence in the control plane, but as far as we know, there is no such theoretical study of the 
affect of the control plane activity on successful packet delivery in the forwarding plane. There are 
empirical papers measuring the successful routing of packets during BGP activity, e.g. [10] , 

There has been a great deal of research concerned with modelling and analysing the convergence 
behaviour of routing protocols like BGP (i.e., the control plane). However, there is not yet such 
a formal study of the behaviour of the forwarding plane. Of course, there are many important 
practical reasons motivating the study of BGP convergence. For instance, convergence is crucial 
for ease of debugging and the predictability of routing. Also, non-convergence can lead to high 
network traffic and high computational load on the routers due to the resulting non-stop additional 
update messages. However, we feel that a formal model and study of the forwarding plane could 
also lead to important insights. While the work in this paper is probably far from having any 
obvious practical implications, we still feel that our work provides some initial steps towards a 
fuller understanding of how the forwarding plane is affected by the choice of routing protocol. 

In a first step towards a theoretical understanding of the affect of the control plane on the 
forwarding plane, we define a family of idealised distributed routing protocols. We base our idealised 
routing protocols on BGP and two particularly important attributes that a routers uses to rank its 
available routes. One such attribute is the set of domains that the route traverses. A router might 
not trust certain domains to handle its packets securely or in a timely fashion so it may reject 
routes traversing such unreliable domains. We call this (no-go) filtering since any route that goes 
through an undesirable domain (i.e., a domain on the router's no-go filtering list) will get filtered 
out of consideration. Secondly, it has been argued that perhaps the most important attribute in 
how a router v ranks routes is the neighbour of v announcing the route to v [16]. That is, one can 
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think of each router ordering its neighbours and ranking any route from a lower ordered router over 
any route from a higher ordered router. This is called next-hop routing. Thus, in our protocols, 
a node ranks routes by first filtering out any that go through nodes on its filtering list and then 
choosing from amongst the remaining routes the one announced by the lowest ordered neighbour 
(next-hop preference with filtering). 

Our main results are that for several versions of this family of protocols, every packet gets to 
their destinations whether or not the control plane has an equilibrium^] 



2 Our Model. 

As discussed, to analyse stability, it suffices to consider only the control plane. But, to understand 
packet routing, we need to understand the interaction between the forwarding and control planes. 
Thus, we need to incorporate the actions of the forwarding plane into the standard model of 
the control plane [3]. To do so, some assumptions must be made, particularly concerning the 
synchronisation between the planes. Because our aim is to obtain performance guarantees that 
apply even in worst-case instances, all our modelling assumptions, with one notable exception, are 
essentially worst-case. 



2.1 Modelling the Forwarding Plane. 

We represent a network by a directed graph G = (V, A) on n nodes. The destination node in the 
network is denoted by a distinguished node r called a sink node. We assume that, for every node 
v € V , there is a directed path in G from v to the sink r, and the sink r has no outgoing arc. At 
any point in time t, each node v chooses at most one of its out-neighbours w as its chosen next-hop; 
thus, v selects one arc (v, w) or selects none. These arcs form a routing graph IZt, each component 
of which is a 1-arborescence, an in-arborescenc^T plus possibly one arc (v,w) emanating from the 
root v of T, i.e, T and TU {(v,uu)} are both 1-arborescences. (If the root of a component does 
select a neighbour, then that component contains a unique cycle.) When the context is clear, for 
clarity of exposition, we abuse the term tree to mean a 1-arborescence, and we use the term forest 
to mean a set of trees. A component (tree) in a routing graph is called a sink- component if it has 
the sink r as a root; other components are called non-sink components. 

Each node selects its outgoing arc according to its preference list of acceptable paths. We 
examine the case where these lists can be generated using two of the most common preference 
criteria in practice: next-hop preferences and filtering. For next-hop preferences, each node v £ V 
has a ranking on its out-neighbours, nodes w such that (v, w) G A. We say that w is the k-th choice 
of v if w is an out-neighbour of v with the k-th rank. For k = 1,2, ... ,n, we define a set of arcs 
Ak to be such that (v,w) £ A^ if w is the k-th choice of w, i.e., A^ is the set of the k-th choice 
arcs. Thus, A\, A2, ■ ■ ■ , A n partition a set of arcs A, i.e., A = A\ U A2 U . . . A n . We call the entire 
graph G = (V,A) an all-choice graph. A filtering list, T>{v), is a set of nodes that v never wants 
its packets to route through. We allow nodes to use filters and otherwise rank routes via next-hop 
preferences, namely next-hop preferences with filtering. 

1 We remark that these results can be viewed in the wider framework of obtaining social welfare guarantees for 
(possibly) non-convergent games in which agents apply best-response dynamics. 

2 An in-arborescence is a graph T such that the underlying undirected graph is a tree and every node has a unique 
path to a root node. 
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To be able to apply these preferences, each node v G V is also associated with a path V(v), 
called v's routing path. The routing path V{v) may not be the same as an actual t>, r-path in 
the routing graph. A routing path V(v) (resp., a node v) is consistent if V(v) is a v, r-path in the 
routing graph; otherwise, we say that V(v) (resp., v) is inconsistent. A node v is clear if the routing 
path V(v) ^ 0, i.e., v has a path to the sink; otherwise, v is opaque. (The node v will never keep 
a path that is not a v, r-path.) We say that a node w is valid for v or is a ua/ic? choice for i> if to 
is clear and V(w) contains no nodes in the filtering list T>{w). If w is a valid choice for u, and v 
prefers w to all other valid choices, then we say that w is the best valid choice of v. A basic step in 
the dynamic behaviour of BGP is that, at any time t, some subset Vt of nodes is activated meaning 
that every node v G Vt chooses the most highest ranked acceptable path V(v) that is consistent 
with one of its neighbours' chosen paths at time t — 1. The routing graph TZt consists of the first 
arc in each routing path at time t. 

Protocol variations result from such things as restricting Vt so that \Vt\ = 1, specifying the 
relative rates that nodes are chosen to be activated and allowing other computations to occur 
between these basic steps. In our protocol, we assume that activation orderings are fair in that 
each node activates exactly once in each time period - a round - the actual ordering however may 
differ in each round. While our protocol is not intended to model exactly the behaviour of BGP, we 
tried to let BGP inspires our choices and captures the essential coordination problem that makes 
successful dynamic routing hard. 

Procedure 1 Activate(u) 
Input: A node v G V — {r}. 

1: if v has a valid choice then 

2: Choose a best valid choice w of v. 

3: Change the outgoing arc of v to (v,w). 

4: Update V(v) := vV(w) (the concatenation of v and V(w)). 

5: else 

6: Update V(v) := 0. 
7: end if 



Procedure 2 Protocol(G,r,7£o) 

Input: A network G = (V, A), a sink node r and a routing graph TZq 
1: Initially, every node generates a packet. 
2: for round t = 1 to . . . do 

3: Generate a permutation nt of nodes in V — {r} using an external algorithm A. 
4: Control Plane: Apply Activate^) to activate each node in the order in irt- This forms a 
routing graph 1Z t ■ 

5: Forwarding Plane: Ask every node to forward packets it has, and wait until every packet 
is moved by at most n hops (forwarded n times) or gets to the sink. 

6: Route- Verification: Every node learns which paths it has in the routing graph, i.e., update 
V(v) := v, r-path in lZ t - 

7: end for 



This entire mechanism can thus be described using two algorithms as follows. Once activated, 
a node v updates its routing path V(v) using the algorithm in Procedure [Tj The generic protocol 
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is described in Procedure [2] This requires an external algorithm A which acts as a scheduler 
that generates a permutation - an order in which nodes will be activated in each round. We will 
assume that these permutations are independent and randomly generated. Our subsequent routing 
guarantees will be derived by showing the existence of specific permutations that ensure all packets 
route. These permutations are different in each of our models, which differ only in filtering lists. We 
remark that our model is incorporated with a route-verification step, but this is not implemented 
in the actual BGP. 

With the model defined, we examine the efficiency of packet routing for the three cases of 
next-hop preferences with filtering: 

• General Filtering. The general case where the filtering list T>(v) of any node v can be an 
arbitrary subset of nodes. 

• Not me ! The subcase where the filtering list of node v consists only of itself, T>(v) = {v}. Thus, 
a node does not want a path through itself, but otherwise has no nodes it wishes to avoid. 

• Anything Goes ! The case where every filtering list is empty, T>{v) = 0. Thus a node does not 
even mind if its packets cycle back through it! 

2.2 Technical Aspects. 

In setting up a model for a practical problem, it is important to examine how the modelling 
assumptions relate to reality. So here we briefly address some technical aspects: 

• Synchronisation of the Planes. Observe that in our model the control plane and the for- 
warding plane operate at a similar speed. This assumption is the worst case in that it maximises 
the rate at which inconsistencies are produced between the nodes routing paths. In practice, 
updates in the control plane are much slower than the rate of packet transfer. 

• Packet Cycling. When a packet gets stuck in a cycle, we will assume that, at the start of the 
next round, an adversary can position the packet at whichever node in the cycle they wish. 

• Fair Activation Sequences. We insist that activation sequences in the control plane are fair 
in that all nodes update their routes at a similar rate. Clearly, the use of permutations ensures 
fairness. From the theoretical point of view, fairness is important as it avoids artificially routing 
packets by the use of unnatural and pathological activation sequences. For example, it prohibits 
the use of activation sequences that are biased towards nodes in regions where disconnectivities 
arise and attempts to fix this by "freezing" other nodes until consistency is obtained. Moreover, 
in practice, routers timings on the control plane are similar. 

• Routing in Rounds. The use of rounds (defined by permutations) for routing is not vital and 
is used for clarity of exposition and to emphasise fairness. Also, packet forwarding is clearly not 
delayed until the end of a "round" in practice but, again, this is also not needed for the model. 
The assumption is made as it clarifies the arguments needed in the analyses. For example, 
forwarding at the end of a round can be shown to be equivalent to forwarding continuously 
throughout the round with the planes in sync; that is, packets are forwarded immediately and, 
within a round, the routing path at a node is updated just before the first packet a node sees 
is about to leave it. 

• Route- Verification. Route-verification at the end of the round is our one non-worst case 
assumption and is not a standard aspect of BGP, albeit one that can be incorporated in a fairly 
simple fashion by tools such as traceroute or an AS-level traceroute tool such as that described 
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by Mao et al. [2]. Route- verification is the focus of the influential paper of John et al. [7] 
on consensus routing. It is also used in the theory literature on incentives under BGP |13j . 
Our results show that the route- verification step can be omitted, but only via the use of unfair 
activation sequences; it remains a open problem to obtain consistency with fair sequences. 
• Filtering. In this paper we assume that each node can apply what is known as import filtering 
- that is, not accepting certain routes from its neighbours. This implicitly assumes that each 
node announces its routing path to all of its neighbours. In reality, each node may choose to 
apply export filtering - that is, it may announce any particular route to only a subset of its 
neighbours (e.g., in order to assure "valley- free routing" [1]). 

Export filtering can be incorporated into our model by allowing for neighbour specific import 
filtering rules, where a node v can have a filtering list T>(v,w) for each neighbour w. Of course 
our lower bounds would still hold for this more general model but it would allow for more special 
cases to explore. 

2.3 Our Results. 

We partition our analyses based upon the types of filtering lists. Our first result is a strong hardness 
result presented in Section [3} Not only can it be hard to determine if every packet can be routed 
but the maximum number of packets that can be routed cannot be approximated well even if the 
network can reach equilibrium. Specifically, 

Theorem 1 . For filtering lists of cardinality at most one, it is NP-hard to approximate the maxi- 
mum stable subtree to within a factor of 0(n 1 ~ e ), for any constant e > 0. 

Corollary 2. For filtering lists of cardinality at most one, given a network in which an equilibrium 
is guaranteed to exist, it is NP-hard to approximate the maximum number of packets that can be 
routed to within a factor of 0(n 1 ~ e ), for any constant e > 0. 

However, for its natural subcase where the filtering list of a node consists only of itself (that is, 
a node doesn't want to route via a cycle!), we obtain a positive result in Section [5] 

Theorem 3. If the filtering list of a node consists only of itself, then an equilibrium can be obtained 
in n rounds. However, every packet will be routed in | rounds, that is, before stability is obtained! 

Interestingly, we can route every packet in the case T>(v) = for all v G V; see Section [4} Thus, 
even if nodes don't care whether their packets cycle, the packets still get through! 

Theorem 4. // the filtering list is empty then every packet can be routed in 4 rounds, even when 
the network has no equilibrium. 

Theorems [3] and [4] are the first theoretical results showing that packet routing can be done in 
the absence of stability. For example, every packet will be routed even in the presence of dispute 
wheels. Indeed, packets will be routed even if some nodes never actually have paths to the sink. 
Note that when we say that every packet will route with probability one we mean that, assuming 
permutations are drawn at random, we will eventually get a fair activation sequence that routes 
every packet. It is a nice open problem to obtain high probability guarantees for fast packet routing 
under such an assumption. 
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3 General Filtering. 



Here we consider hardness results for packet routing with general filtering lists. As discussed, 
traditionally the theory community has focused upon the stability of 72. — the routing graph is 
stable if every node is selecting their best valid neighbour (and is consistent). For example, there 
are numerous intractability results regarding whether a network has an equilibrium. However, that 
the routing graph may be stable even if it is not spanning! There may be singleton nodes that prefer 
to stay disconnected rather than take any of the offered routes. Thus, regardless of issues such as 
existence and convergence, an equilibrium may not even route the packets. This can be particularly 
problematic when the nodes use filters. Consider our problem of maximising the number of nodes 
that can route packets successfully. We show that this cannot be approximated to within a factor of 
re 1-6 , for any e > unless P = NP. The proof is based solely upon a control plane hardness result: 
it is NP-hard to approximate the maximum-size stable tree to within a factor of n 1_<E . Thus, even if 
equilibria exist, it is hard to determine if there is one in which the sink- component (the component 
of 1Z containing the sink) is large. 

Formally, in the maximum-size stable tree problem, we are given a directed graph G = (V, E) 
and a sink node r; each node v S V has a ranking of its neighbours and has a filtering list T>(v). 
Given a tree (arborescence) T C G, we say that a node u is valid for a node v if (u, v) S E and a 
u,r-path in T does not contain any node of T>{v). We say that T is stable if, for every arc (u,v) 
of T, v is valid for u, and u prefers v to any of its neighbours in G that are valid for u (w.r.t. T). 
Our goal is to find the stable tree (sink-component) with the maximum number of nodes. We will 
show that even when |7J?(v)| = 1 for all nodes v £ V, the maximum-size stable tree problem cannot 
be approximated to within a factor of n 1 ^ 6 , for any constant e > 0, unless P = NP. 

The proof is based on the hardness of 3 SAT given a CNF-formula on N variables and M 
clauses, it is NP-hard to determine whether there is an assignment satisfying all the clauses. Take 
an instance of 3SAT with N variables, x\, X2, ■ ■ ■ , xjy and M clauses C\, C2, . . . , Cm- We now create 
a network G = (V, A) using the following gadgets: 

• Var-Gadget: For each variable xi, we have a gadget H(xi) with four nodes a,i,uj ,uf ,b{. 
The nodes uf and uf have first-choice arcs (uj,ai), (uf , etj) and second-choice arcs (uf,bi), 
(uf,bi). The node a, has two arcs (ai,uf) and (oj,uf); the ranking of these arcs can be 
arbitrary. Each node in this gadget has itself in the filtering list, i.e., T>{y) = {v} for all nodes 
v in H(xi). 

• Clause- Gadget: For each clause Cj with three variables x^n , x^) 5 x i($) 5 we have a gadget 
Q(Cj). The gadget Q(Cj) has four nodes Sj,qij,q2j,q3,j,tj. The nodes qij, q^j, qs t j have 
first-choice arcs (qij,tj), (q2,j,tj), (Q3,j,tj). The node Sj has three arcs (sj,qij), (sj,<l2,j)i 
(sj,q3j); the ranking of these arcs can be arbitrary, so we may assume that (sj,q z ,j) is a 
zth-choice arc. Define the filtering list of Sj and tj as T>{sj) = {sj} and T>(tj) = {do}- (The 
node do will be defined later.) For z = 1,2,3, let uj^ be a node in a Var-Gadget H(x i ^); 
the node q z j has a filtering list T>(q z j) = {uj^}, if assigning x^ = True satisfies the clause 
Cj; otherwise, T>(q x j) = {uf^}. 

To build G, we first add a sink node r and a dummy sink do; we connect do to r by a first-choice 
arc (do, r). We arrange Var-Gadgets and Clause-Gadgets in any order. Then we add a first-choice 
arc from the node a\ of the first Var-Gadget H(x\) to the sink r. For i = 2,3, ... ,N, we add a 
first-choice arc 04—1) joining gadgets H(xi^i) and H(xi). We join the last Var-Gadget H(xn) 
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and the first Clause-Gadget Q(C\) by a first-choice arc (ti,ajy). For j = 2,3, .. . ,M, we add a 
first-choice arc (tj,Sj-\) joining gadgets Q(Cj-i) and Q(Cj). This forms a line of gadgets. Then, 
for each node q z j of each Clause-Gadget Q(Cj), we add a second-choice arc (q z j,do) joining q z j 
to the dummy sink do. Finally, we add L padding nodes di, d2, ■ ■ ■ , &l and join each node d%, for 
% = 1,2, ... ,L, to the last Clause-Gadget Q(cm) by a first-choice arc (dj, sjvf); the filtering list 
of each node dj is T>(di) = {do}, for all i = 0, 1, . . . , L. The parameter L can be any positive 
integer depending on a given parameter. Observe that the number of nodes in the graph G is 
AN + 5M + L + 2, and |£>(v)l = 1 f° r an nodes v of G. The reduction is illustrated in Figure [TJ 




Figure 1: The hardness construction. 



The correctness of the reduction is proven in the next theorem. 

Theorem 5. For any constant e > 0, given an instance of the maximum-size stable tree prob- 
lems with a directed graph G on n nodes and filtering lists of cardinality \T>{v)\ = 1 for all nodes 
v, it is NP-hard to distinguish between the following two cases of the maximum-size stable tree 
problem. 

• Yes-Instance: The graph G has a stable tree spanning all the nodes. 

• No-Instance: The graph G has no stable tree spanning n € nodes. 

Proof. We apply the above reduction from 3SAT with a parameter L = J 1//e — J, where J = 
An + 5m + 2. Thus, the graph G has n = J l / e nodes and has n e = J non-padding nodes. 

First, we show that there is a one-to-one mapping between choices of each Var-Gadget H(xi) 
and an assignment of X{. Consider any Var-Gadget H(xi). To connect to the next gadget, nodes 
uf and uf of H{xi) must choose at least one second-choice arc. However, in a stable tree, they 
cannot choose both second-choice arcs (uf,bi) and (uf,bi); otherwise, uf or uf would prefer to 
choose the node o&j. Thus, the gadget G(xi) must choose either arcs 

(1) (uj,bi), (uf ,ai), (ai,uf) or (2) (uf , hi), {uf , a { ), (a h uf ). 

These two cases correspond to the the assignments x% = True and Xi = False, respectively. Thus, 
there is a one-to-one mapping between the choices of gadget H(xi) in the stable tree and the 
assignment of Xj. We will refer to these two possibilities of a stable solution as an assignment of X{. 
Now, we prove the correctness of the reduction. 

Yes-Instance: Suppose there is an assignment satisfying all the clauses. Then, by the above 
discussion, there is a stable tree T corresponding to such assignment. We choose an arc (do,s) 
and all the arcs connecting two gadgets (or from the first Var-Gadget to the sink r). For each 
Clause-Gadget Q(Cj), we choose a first-choice arc (qz,ji ^j)> ^ — ^> ^> ^> ^ the assignment to Xi( z ^ 
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satisfies Cj\ otherwise, we choose a second-choice arc (q z ,j, do)- For the node Sj of Q(Cj), we choose 
an arc (sj,q z j), where z is the smallest number such that the assignment to x^ satisfies Cj (i.e., 
q z j chooses tj); since the given assignment satisfies all the clauses, Sj has at least one valid choice. 
Now, we have that the node sm of the last Clause-Gadget Q(Cj) has a path V{sm) to the sink r 
that does not contain the dummy sink do- Thus, every padding node can choose sm an d is in the 
stable tree T, implying that T spans all the nodes. 

No-Instance: Suppose there is no assignment satisfying all the clauses. Let T be any stable 
tree of G. As in the previous discussion, the choices of nodes in Var-Gadgets correspond to the 
assignment of variables of 3SAT. 

Consider any Clause-Gadget Q(Cj). Since T>(sj) = {do}, the node Sj of Q(Cj) has a path to 
the sink r only if 

(1) a iy,r-path V(tj) in T does not contain the dummy sink do, and 

(2) one of qij, q2j, qz,j chooses tj. 

These two conditions hold only if T corresponds to an assignment satisfying Cj. To see this, 
suppose the first condition holds. Then V(tj) has to visit either vf or vf of every Var-Gadget 
H{xi), depending on the assignment of Xi. Thus, by the construction of V(q z j), tj is valid for q z j 
only if the assignment to Xi> z \ satisfies Cj. Since there is no assignment satisfying all the clauses, 
a node S£ of some Clause-Gadget Q{Cg) is not in T. This means that nodes in the remaining 
Clause-Gadget have to use the dummy sink do to connect to the sink r. Thus, the node sm of the 
last Clause-Gadget Q{Cm) is not in T and so are all padding nodes d\,d2, ■ ■ ■ ,d^. Therefore, the 
size of T is at most J = rf, proving the theorem. □ 

From the perspective of the nodes, it is NP-hard to determine whether adding an extra node 
to its filtering list can lead to solutions where none of its packets ever route. In other words, it 
cannot avoid using intermediate node it dislikes! 

4 Filtering: Anything- Goes! 

Here we consider the case k = 0, where every node has an empty filtering list. This case is 
conceptually simple but still contains many technical difficulties involved in tracking packets when 
nodes become mistaken in their connectivity assessments. In this case, networks with no equilibrium 
can exist. 

Figure [2] gives an example in which no stable spanning tree exists when the filtering lists are 
empty (that is, k = 0). Moreover, there exist fair activation sequences such that the node v will 
never be in the sink component; for example, repeatedly active nodes according to the permutation 
(v, u, w, x, y). Despite this every packet will route in two rounds! This example illustrates the need 
to track packets if we want to understand the efficacy of BGP. 

We show, however, that even in such circumstances, every packet still reaches the sink, and this 
is the case for all networks. Specifically, we present a fair activation sequence of four rounds that 
routes every packet, even when there is no equilibrium. 

For k = 0, a node v only needs to known whether its neighbour u has a path to the sink since 
v does not have any node it dislikes. Thus, we can view each node as having two states: clear or 
opaque. A node is clear if it is in the routing-tree (the nomenclature derives from the fact that 
a packet at such a node will then reach the sink - that is, "clear"); otherwise, a node is opaque. 
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Figure 2: A network with no stable spanning tree. (Arc numbers indicate rankings, e.g., the number 
2 the edge (u,x) means that x is the second choice of u.) 



Of course, as nodes update their chosen next-hop over time, they may be mistaken in their beliefs 
(inconsistent) as the routing graph changes. In other words, some clear nodes may not have "real" 
paths to the sink. After the learning step at the end of the round, these clear-opaque states are 
correct again. 

Our algorithm and analysis are based on properties of the network formed by the first-choice 
arcs, called the first class network. We say that an arc (u, v) of G is a first-choice arc if v is the 
most preferred neighbour of u. We denote the first class network by F = (V,Ai), where A\ are 
the first-choice arcs. As in a routing graph 1Z , every node in F has one outgoing arc. Thus, every 
component of F is a 1-arborescences, a tree-like structure with either a cycle or a single node as 
a root. We denote the components of F by Fq, Fi, . . . , Ft, where Fq is the component containing 
the sink r. Observe that, when activated, every node in Fq will always choose its neighbour in Fq. 
So, we may assume wlog that Fq is a singleton. Each Fj has a unique cycle Cj, called a first class 
cycle (We may assume the directed cycle in Fq is a self- loop at the sink r.) The routing graph at 
the beginning of Round t is denoted by TZ t ■ We denote by K, t and Ot the sets of clear and the set 
of opaque nodes at the start of Round t. Now, we will show that there is an activation sequence 
which routes every packet in four rounds. 

The proof has two parts: a coordination phase and a routing phase. In the first phase, we give 
a coordination algorithm that generates a permutation that gives a red-blue colouring of the nodes 
with the following three properties: (i) For each Fj, every node in Fj has the same colour, i.e., 
the colouring is coordinated, (ii) If the first class cycle Cj of Fj contains a clear node then all 
nodes in Fj must be coloured blue, (iii) Subject to the first two properties the number of nodes 
coloured red is maximised. The usefulness of this colouring mechanism lies in the fact that the 
corresponding permutation is a fair activation sequence that will force the red nodes to lie in the 
sink-component and the blue nodes in non-sink-components. Moreover, bizarrely, running this 
coordination algorithm four times in a row ensures that every packet routes! So in the second 
phase, we run the coordination algorithm three more times. 

4.1 Coordination Phase. 

The algorithm Coordinate(/C i) in Procedure [3] constructs a red-blue colouring of the nodes, i.e. the 
final partition (R,B) of V. At the termination of Coordinate (JC t), by the construction, any node 
v E R prefers some node in R to any node w E B, and any node v E B prefers some node in B to 
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Procedure 3 Coordinate (/C t) 
Input: A set of clear nodes K, t- 
Output: A partition {R, B) of V. 

1: Let Bq := LW(C)n/e t ^0 ^(-^0 ' 3e a set °^ n °des containing in an F-component whose first class 
cycle Ci has a clear node. 

2: Initialise q := 0. 

3: repeat 

4: Update g := q + 1. 

5: Initialise B q := B q -i, R q := {r} and U := V - (R q U B q ). 

6: while 3 a node v e U that prefers a node in R q to nodes in F g U (U D F t ) do 
7: Move f from U to R q . 
8: end while 

9: while 3 a node v E U that prefers a node in F g to nodes in R q U (U n F t ) do 
10: Move w from U to B q . 
11: end while 

12: Move UnJCt from U to B q . 

13: until 2? 9 = 

14: return (R q ,B q ). 



any node w £ R. 

Given a partition (R,B), we generate an activation sequence as follows. First, we greedily 
activate nodes in R — {r} whenever their most-preferred clear neighbours are in R. (We activate 
nodes of R in the same order as we constructed R.) This forms a sink-component on R. Next, we 
activate nodes in B. We start by activating nodes in Bq = UrG^n-ft^S ~ the components of 

F whose first class cycles contain at least one clear node. For each Fi with V(Fi) C Bq, take a 
clear node v G Cj D /C t- Then activate the nodes of Fi (except v) in an increasing order of distance 
from v in Fj, and after that we activate v. This forms a non-sink-component F in the routing 
graph as every node can choose its first-choice. Finally, we activate nodes in B — Bq whenever their 
most-preferred clear neighbours are in B (we use the same order as in the construction of B). This 
creates non-sink-components on B and implies the next lemma. 

Lemma 6. Let i\ t be an activation sequence generated from (R, B) as above. At the end of the 
round, the following hold: 

• The sink- component includes R and excludes B. 

• Coordination: For each Fi, either all the nodes of Fi are in the sink- component or none of 
them are. 

• Let Bq = Ury(c»)nJC t ^0 o-nd suppose ICt = Bq. If a packet travels for n hops but does 
not reach the sink, then it must be at a node in ICt ■ 

Proof. The first statement follows from the construction. For the second statement, it suffices to 
show that, for each Fi, either V(Fj) C R or V(Fi) C B. Suppose not. Then there is V{Fi) crossing 
R or B. But, then some node in R (respectively, B) would have a first-choice in B (respectively, 
R), and this is not possible by the construction of (R, B). 

For the third statement, note that a packet that travels for n hops but does not reach the sink 
must be stuck in some cycle. Consider the construction of (R,B). Since ICt = Bq, we only add 
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node to B whenever it prefers some node in B to any node in R. Because U PI ICt = 0, nodes in 
B — Bo cannot create a cycle on their own. Thus, the packet is stuck in a cycle that contains a 
clear node; the only such cycles are the first class cycles of Bq since ICt = Bq. □ 

The following lemma follows by the construction of a partition (R, B). 

Lemma 7. Let (R',B') be any partition generated from the procedure Coordinate(.), and let (R, B) 
be a partition obtained by calling Coordinate (ICt) with ICt Q B' . Then R' C R. 

Proof. Consider a partition (R q ,B q ) constructed during a call to Coordinate (JC t). Observe that 
Bq C B' because Bq = Ui-y((7,-)nJCt^0 ^"C^O ana - Lemma |6j implies that each F\ is contained in 
either R' or B' . By the construction of (R',B'), since Bq C B', every node of R' must have been 
added to R±, i.e., R' C Inductively, if i?' C for some q > 1, then 5 g C 5' and thus 
-R' ^ -Rg+i by the same argument. □ 

4.2 Routing Phase: A Complete Routing in Four Rounds. 

Running the coordination algorithm four times ensures every packet will have been in the sink- 
component at least once, and thus, every packet routes. 

Theorem 8. In four rounds every packet routes. 

Proof. The first round t = 1 is simply the coordination phase. We will use subscripts on R and 
B (e.g., Rt and Bt) to denote the final colourings output in each round and not the intermediate 
sets R q /B q used in Coordinate (.). Now, consider a packet generated by any node of V . First, we 
run Coordinate (/C i) and obtain a partition (R\,Bi). By Lemma [6j if the packet is in R±, then it 
is routed successfully, and we are done. Hence, we may assume that the packet does not reach the 
sink and the packet is in B\. Note that, now, each Fi is either contained in Ri or B\ by Lemma |6| 

We now run Coordinate (/C 2) and obtain a partition (R 2 ,B 2 ). By Lemma [6j IC2 = R\- So, if 
the packet does not reach the sink, it must be in B2. Since no F-component crosses Ri, we have 
Ri = /C 2 = Ui:V(Ci)n)C2^0 So, Ri Q B 2 (since JC 2 Q B 2 ) and R 2 C B x , and Lemma | 

implies that the packet is in R\. 

Third, we run Coordinate (IC 3) and obtain a partition (i?3,i?3). Applying the same argument 
as before, we have that the packet is in R 2 (or it is routed), R 2 C B3 and i?3 C B 2 . Now, we 
run Coordinate(/C 4) and obtain a partition {R,4,B^). Since R3 = /C4 C B 2 , Lemma [7] implies that 
R 2 C R 4 . Thus, the packet is routed successfully since R4 is contained in the sink-component. 

□ 

5 Filtering: Not-Me! 

In practice, it is important to try to prevent cycles forming in the routing graph of a network. 
To achieve this, loop-detection is implemented in the BGP-4 protocol [17J. The "Not-Me" filtering 
encodes loop-detection in the BGP-4 protocol simply by having a filtering list T>(v) = {v}, for every 
node v. In contrast to Theorem [5j which says that it is NP-hard to determine whether we can 
route every packet, here we show that every packet will route. Moreover, we exhibit a constructive 
way to obtain a stable spanning tree via fair activation sequences. Interestingly, all of the packets 
will have routed before stability is obtained. In particular, we give an algorithm that constructs an 
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activation sequence such that every packet routes successfully in rounds, and the network itself 
becomes stable in n rounds. This is the most complicated part of our paper, and the algorithm 
on its own is very long to describe. Thus, we will first give a high level overview and defer the 
algorithm and its proof to later parts of this section. 

Overview: When filtering lists are non-empty, a complication arises since even if w is the most 
preferred choice of v and w has non-empty routing path, v still may not be able to choose w. This 
makes the routing graph hard to manipulate. The key idea is to manipulate the routing graph a 
little-by-little in each round. To do this, we find a spanning tree with a Strong Stability Property. 

Property 9 (Strong Stability Property (Informal)). A spanning tree S has the strong stability 
property on O if, for every node v G O, the most preferred choice of v is its parent w, even if it 
may choose any node except those that are descendants and in O. Thus, if we activate nodes of S 
in increasing order of distance from the sink r, then every node v G O will always choose w. 

It is easy to find a stable spanning tree, a tree where no node wants to change its choice, and 
given a stable spanning tree S, it is easy to force opaque nodes in Ot to make the same choices 
as in S. But, this only applies to the set of opaque nodes, so it may not hold in the later rounds. 
The strong stability property allows us to make a stronger manipulation. Intuitively, the strong 
stability property says that once we force every node v G O to make the same choice as in S, we 
can maintain these choices in all the later rounds. Moreover, in each round, if we cannot route all 
the packets, then we can make the strong stability property span three additional nodes; if so, the 
property spans one more node. Thus, in rounds, every packet will route, but we need n rounds 
to obtain stability. 

Definitions and Notations. Before proving the main result, we need some definitions and 
notations. We denote by Oi and K, i the set of opaque and clear nodes, respectively, at the beginning 
of Round i. Given a graph R and a set of nodes U, we denote by R[U] = {(u, v)) : u,v G U, {u, v) G 
R} a subgraph of R induced by U, and we denote by = {(u, v) : u G U, (u, v) G R}, a subgraph 
of R induced by U plus the set of arcs leaving U. Given a set of nodes Q C V, the Q-subtree 
of v (with respect to T) is a (maximal) subtree of a tree (or forest) T with v as a root node. A 
(spanning) tree T is stable if every node v with (v, w) G T, prefers its parent w in T to every node 
not its descendants; thus, no node wants to changes its next hop. A spanning tree T has the strong 
stability property on the set of nodes O if every node v G O with (v, w) G T prefers its parent w to 
every node outside its O-subtree; observe that if O = V (or V — {r}), then T is also stable. 

5.1 Find a Tree with The Strong Stability Property 

In this section, we present a subroutine for finding a spanning tree with the strong stability property. 
The input of this algorithm is a sink-component T m , a spanning tree S m with the strong stability 
property on a given set of nodes O. Then the algorithm extends the strong stability property to 
hold on a set O of nodes not in the sink-component T m . Our algorithm is presented in Procedure [4| 
The next lemma shows the correctness of the procedure Stabilize(T m , S m ,Q). 

Lemma 10. The procedure Stabilize(T in , S m , O) outputs a spanning tree with the strong sta- 
bility property on O U O. 
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Procedure 4 FindStable(T in , S in ,Q) 



Input: A sink-component T m and a spanning (or empty) tree S m such that 

(1) The tree S m has the strong stability property on O, and 

(2) 4,„(0)CS m . 

Output: A stable spanning tree 5°"* with the strong stability property on OUO, where O = 
V - V{T in ) is the set of nodes not in T in . 

1: Let O = V — V(T m ) be a set of nodes not in the sink component T m . 

2: Initialise S out := T in U A% n (0). 

3: Initialise d := T in [0 - O] and C := 0. 

4: for iteration t = 1 to \0 - 0| do 

5: Pick an arbitrary leaf v of Ct- 

6: Pick a node w G y(S' on *) such that u prefers u; to other nodes not in its ©-subtree in S out . 

7: Replace an arc (v,y) in S out by an arc (v,w). 

8: Update C t := C t -i U {v}. 

9: Update C m :=C t - {«} = V(d) - C t . 
10: end for 
11: return S out . 



Proof. First, we have to show that the graph 5°"* is a tree before getting into the loop. Since T m 
has no arc going to O, adding arcs of Aa in (0), which is a forest, to T m cannot create a cycle. Every 
node u G O has a directed path to sink r by using a path in 5°"* to some node in T m and then 
using a path in T m to the sink. Thus, initially, S" 3 '"* is a spanning tree. By the construction, the 
spanning tree S"™* has the strong stability property on O, and this holds throughout the procedure 
because we never change outgoing arcs of nodes in O. Next, consider the loop in the procedure. 
By the choice of the node w chosen by v, w is not a descendant of v in S out . Thus, replacing the 
arc (v,y) G S" 5 "* by the arc (v,w) never creates a cycle, implying that S out is always a spanning 
tree. It remains to show that S out has the strong stability property on Ct m every iteration t. We 
prove this by induction on t. This is true for t = 1 as Cq = 0. Now, consider iteration t, and 
assume that the strong stability property holds on Ct-i- Since v is a leaf of Ct, every node in the 
{O — 0)-subtree of v (w.r.t. i> out ) is in C t-i, which means that it will not change its parents. Thus, 
the strong stability property holds on v by the choice of w, completing the proof. □ 

5.2 Routing Every Packet in n Rounds. 

In this section, we present the main algorithm that routes every packet in n rounds (recall, each 
round consists of a single fair-activation sequence). Beside the procedure FindStable(. ,.,.), a breath- 
first-search (BFS) algorithm is our basic building block for generating an activation sequence. Given 
a set spanning tree F and a set of nodes U C V, the procedure BFS(C7, F) (see Procedure]^ activates 
the nodes of U in breadth- first- search (BFS) order with respect to the tree F; that is, we activate 
the nodes of U in increasing order of distance to the sink node r in F. If we activate these nodes 
in the reverse BFS order, then we will denote the procedure by reverse-BFS(C/, F). 

Over the course of these n rounds, we will work with the following two classes of nodes. First, 
let Kt = n* =1 /Cj be the set of nodes that have been clear in every round up to time t. Second, 
consider its complement Q t = U* =1 Oj, the set of nodes that have been opaque at least once by time 
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Procedure 5 BFS(f/,F) 

Input: A set U C V and a spanning tree -F. 

1: Let vi, V2, ■ ■ ■ , v g be nodes in £7 — {r} sorted in increasing order of distance to the root v of F. 

2: for i := 1 to q do 

3: Activate t>j. 

4: end for 



Procedure 6 Fair-Stabilize() 
1: Let Tt be the sink-component at the beginning of round t. 
2: Initialise So := an arbitrary spanning tree, Kq = V and Do = 0. 
3: for round t : = 1 to n do 

4: Apply FindStable(Tt, <St_i,Ot_i) to compute a spanning tree St- 

5: Update O t := O t _i U O t and K t := V - O t . 

6: Activate BFS(O t ,S t ). 

7: Activate reverse-BFS(K t , St). 

8: Let v be the first node activated in the previous step. 

9: Let (v, w) be an arc chosen by v in the routing graph. 
10: Replace an arc (y, y) in St by the arc (v, w), and update Ot := Ot U {v}. 
11: end for 



First, it can be seen that this algorithm is fair because Kt and Of partition the set of nodes. 
Now, we will provide some intuition behind the method. Observe that if we can make our routing 
graph T t+ i looks like the spanning tree St, then every packet will route. Typically, though, any 
activation sequence that attempts to do this will induce inconsistencies. This, in turn, will force 
nodes to go opaque. But, it turns out that we can make the nodes in Of choose arcs in accordance 
with St via the use of BFS(Of , St). (It is not at all obvious this can be done because nodes in Ot 
may actually be clear, that is, they need not be in Ot-) 

Then the question is how do we keep track of the packets. The key point is that nodes in 
Ot choose arcs in accordance with the spanning tree St- Since Oi C O2 C O3 C • • • and the 
containments are strict, we will eventually have Ot = V, and our routing graph will be a spanning 
tree. Thus, every packet routes! Moreover, the strong stability property of St on Ot = V also 
implies that the final routing graph is a stable spanning tree. 

The following lemma will then give the key property we need for the proof of the main theorem. 

Lemma 11 (Key Lemma). Consider the end of the t-th round of Fair- Stabilize (). The spanning 
tree St has the strong stability onO t , and for each node v G Ot, if(v,w) G St, then upon activation, 
v chooses w as its next hop. Thus, A~g (Ot) is contained in the routing graph at the end of round. 
(Note that this is before the learning phase.) 

Proof. We proceed by contradiction. Assume that round t is the first time such that one of the two 
statements does not hold. Thus, at the end of round t, either 

(1) the spanning tree St does not has the strong stability property on Ot or 

(2) Ag (Ot) is not contained in the routing graph. 
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First, by the assumption of t, the procedure FindSt able (.,.,.) has the required input and thus 
outputs a spanning tree St with a strong stability property on Of (before adding v). 

Next, we prove that after applying BFS(Ot, St), the routing graph looks exactly like St- This is 
true for all nodes outside Ot by the construction of St- Suppose there is a node p with an arc (p, q) 
in S, but p does not choose q as its next hop. Again, we assume that p is the first such node. So, q 
is clear at this point. Recall that Ag _ (Ot-i) is contained in the routing graph at the end of the 
previous round (before the learning phase). Thus, Aj>(Ot-i) C Ag (Ot-i). Since Ot = Ot-% UOt 
where Ot is the set of opaque nodes, we conclude that Aj- (Ot) C Ag (Ot). Hence, if p is opaque, 
then all descendants of p in the Ot-subtree of p (w.r.t. St) are opaque. This implies that q is the 
most preferred choice of p by the strong stability property of St, a contradiction. If p is clear, then 
the arc (p, q) is in T t . So, every node x that has p in its routing path V(x) also has q in its routing 
path, which means that q does not has p in its routing path. Then, by the strong stability property 
of St, q is the most preferred choice of p, again a contradiction. Thus, the routing graph now looks 
exactly like St- Moreover, every node is consistent since no node changes its next hop. 

Now, consider the node v. By the choice of v, the node v has no descendant in K w.r.t. St- 
Since the routing graph now looks exactly like St and every node is consistent, v can choose any 
node outside its Ot-subtree w.r.t. St- Therefore, after updating St and Ot, it can be seen that 
St has the strong stability property on Ot, and (v,w) is in both St and the routing graph, which 
means that both two properties hold, a contradiction. This completes the proof. □ 



Lemma 1 1 implies the next lemma, which gives a guarantee on the increasing of the size of • 



Lemma 12. Consider the t-th round of Fair- Stabilize () . If some packet does not route in the 
previous round, then \Ot\ > |Of | +3. Otherwise, \Ot\ > |©t| + 1. 

Proof. By the construction, we have \Ot\ > \Ot\ + 1. Thus, the second statement clearly holds. 
Now, we prove the first statement. It suffices to show that if there is an opaque node in Ot, then 
there are at least two opaque nodes not in Ot-i, implying that we add at least three nodes to Ot. 

Assume a contradiction that \Ot — Qt_i| = 1. Let v be a node in Ot — Ot-i- Then, at the 
end of rounds t — 1, there is a cycle C in the routing graph that contains v and other nodes in 
C are in Ot-i- Observe that \C\ > 3 because no node can choose its children. Let a be a node 



that chose v and w be a node chosen by v. By Lemma 11, the routing graph at the end of round 
t — 1 (before the learning phase) contains Ag (Ot_i), which is a forest. Moreover, after applying 
BFS(Ot-i, St—i), every node in Ot— i is consistent because no node changes its next hop. Thus, 
both u and w have v in its chosen routes, implying that w is not valid for v, a contradiction. This 
completes the proof. □ 



It is immediate from Lemma 12 that we can route every packet in [n/3\ rounds, and the network 
becomes stable in n rounds. Moreover, we can deduce a stronger failure guarantee. Let say that the 
round t is a failed-round if we cannot route some packet. We will have at most \n/3\ failed-rounds 
(may not be consecutive rounds) although the routing graph is not yet stable. 

Theorem 13. There is an activation sequence that routes every packet in [n/3\ rounds, gives a 
stable spanning tree in n rounds and guarantees that there are at most [n/3j failed-rounds. 
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5.3 Consistency with Unfair Activation Sequences. 



In this section, we show that, given a network with inconsistent states, there is an unfair activation 
sequence that leads to all consistent states. For each node v, we denote by Vt(v) a routing path 
of v at time t. We remark that, here, activations are not done in rounds. Thus, we assume that 
each time t, we activate one single node. Recall that a node v is consistent if either V(v) = or 
V(v) C lZ t , where TZt is the routing graph at time t. A routing path V(v) is not valid if v G V(v) 
or some node appears more that once in V(v). 

Lemma 14. For any initial state, there exists an unfair activation sequence that produces a state 
where all nodes are consistent. 

Proof. Define B t C V so that v G B t if and only if Vt(v) is not consistent with the routing graph 
TZt (i-e., Vtiv) is a u,r-path in 1Z t or Vt(v) = 0). A node v G B t is called a 6ad node. 

We now define an activation sequence a as follows. The node that will be activated by a at 
the beginning of time step t is some node v where ('Pt(v)l < 1^(^)1 for all u G B t . Thus, when v is 
activated, v will do one of the following: 

(1) The node v will choose a node u and set Vt+i(v) = uPt{u), for some u G B t , where Vt(u) is at 
least as long as Vt(v). 

(2) The node v has no valid choice and will set V v (t + 1) = 0, i.e., for every out-neighbour u of v, 
either v G Vt(u) or Vt(u) = 0. 

(3) The node v will choose a node u and set V v (t + 1) = uV u {t) where u g" B t , i.e., Vt(u) C TZt- 

If case (1) holds, then \Vt+i(v)\ > \Vt(v)\. If case (2) or case (3) holds, then v is removed from 
the set of bad nodes. Observe that for "Not-Me" filtering, there is no node w that appears twice 
in Vt{u). Otherwise, we would have that w G Vt{w) C Vt(u). Thus, |"Pt(^)| < n for all nodes v at 
any time t. It follows that a particular node v cannot be activated more than n times before it is 
either removed from the set of bad nodes or its routing path Vt(v) is such that \Vt(v)\ = n + 1. 
The latter case is not possible since \Vt{v)\ < n. Therefore, in at most n 2 activations, every node 
becomes consistent. □ 
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